Error Analysis of Generalized Nyström Kernel Regression
نویسندگان
چکیده
Nyström method has been successfully used to improve the computational efficiency of kernel ridge regression (KRR). Recently, theoretical analysis of Nyström KRR, including generalization bound and convergence rate, has been established based on reproducing kernel Hilbert space (RKHS) associated with the symmetric positive semi-definite kernel. However, in real world applications, RKHS is not always optimal and kernel function is not necessary to be symmetric or positive semi-definite. In this paper, we consider the generalized Nyström kernel regression (GNKR) with `2 coefficient regularization, where the kernel just requires the continuity and boundedness. Error analysis is provided to characterize its generalization performance and the column norm sampling strategy is introduced to construct the refined hypothesis space. In particular, the fast learning rate with polynomial decay is reached for the GNKR. Experimental analysis demonstrates the satisfactory performance of GNKR with the column norm sampling.
منابع مشابه
Ensemble Nystrom Method
A crucial technique for scaling kernel methods to very large data sets reaching or exceeding millions of instances is based on low-rank approximation of kernel matrices. We introduce a new family of algorithms based on mixtures of Nyström approximations, ensemble Nyström algorithms, that yield more accurate low-rank approximations than the standard Nyström method. We give a detailed study of va...
متن کاملNystrom Approximation for Sparse Kernel Methods: Theoretical Analysis and Empirical Evaluation
Nyström approximation is an effective approach to accelerate the computation of kernel matrices in many kernel methods. In this paper, we consider the Nyström approximation for sparse kernel methods. Instead of relying on the low-rank assumption of the original kernels, which sometimes does not hold in some applications, we take advantage of the restricted eigenvalue condition, which has been p...
متن کاملDouble Nyström Method: An Efficient and Accurate Nyström Scheme for Large-Scale Data Sets
The Nyström method has been one of the most effective techniques for kernel-based approach that scales well to large data sets. Since its introduction, there has been a large body of work that improves the approximation accuracy while maintaining computational efficiency. In this paper, we present a novel Nyström method that improves both accuracy and efficiency based on a new theoretical analy...
متن کاملScalable Kernel K-Means Clustering with Nystrom Approximation: Relative-Error Bounds
Kernel k-means clustering can correctly identify and extract a far more varied collection of cluster structures than the linear k-means clustering algorithm. However, kernel kmeans clustering is computationally expensive when the non-linear feature map is highdimensional and there are many input points. Kernel approximation, e.g., the Nyström method, has been applied in previous works to approx...
متن کاملError Analysis for Direct Linear Integral Equation Methods*
An error analysis of projection methods for solving linear integral equations of the second kind is presented. The relationships between several direct methods for solving integral equations are examined. It is shown that the error analysis given is applicable to other methods, including a modified Nyström method and certain degenerate kernel methods.
متن کامل